Gesture Based Educational Software for Children with Acquired Brain Injuries

“GESBI” is gesture based audio visual teaching tool designed to help children with acquired brain injuries, providing hours of entertainment in a play-and-learn environment while introducing the foundation skills in basic arithmetic, spelling, reading and solving puzzles. These children communicate with the computer via gestures based on my previous research paper “KONCERNHand Gesture Recognition for Physically Impaired” in which gestures are captured by camera and processed without the need of wearing any sensor based gloves etc. KeywordsAcquired Brain Injury, Educational Software, disabilities, gestures. I. ACQUIRED BRAIN INJURY Acquired Brain Injury (ABI), head injury, or acquired brain damage (ABD) is used to describe all types of brain damage, which occur after birth. The brain controls every part of human life: physical, intellectual, behavioral, social and emotional. When the brain is damaged, some part of a person's life will be adversely affected. Even a mild injury can sometimes result in a serious disability that will interfere with a person’s daily functioning and personal activities for the rest of their life [1]. There is very little understanding or knowledge in the community about brain injury and the impact it has on individuals. Children with a brain injury may have difficulty controlling, coordinating and communicating their thoughts and actions but they usually retain their intellectual abilities. It usually affects cognitive, physical, emotional, social or independent functioning [2]. In an effort to identify specific functional deficits that often occur following an Acquired Brain Injury, this paper will consider only impairment related to expression, which prevents the student from being able to speak words even though they are aware of what they want to say. Speech may also be affected by poor cognitive motor function (apraxia) [3]. In this case, the student is unable to make the sounds necessary to form speech. Writing may be affected by the difficulty in producing legible handwriting with age-appropriate speed. II. THE DIFFICULTY IN COMMUNICATION WITH ACQUIRED BRAIN INJURY. Most of children lead highly structured lives: they wake up at the same time, follow the same pattern in morning hygiene, eat meals at the same time, and work the same hours each day. This kind of structure allows them to put most of their lives on automatic pilot and reserve creativity, memory, and novelty for more important areas. Far too often, Brain injured children have no structure in their daily lives and therefore accomplish very little each day: they nap throughout the day and then can't sleep at night; they eat meals at varying times and therefore can't recall if they have eaten at all; they leave things wherever they please and then can't find them. Tight structure reduces the need to continually make decisions, vastly increases the capabilities of the injured children, and significantly reduces the demands placed upon the caregiver [4]. Although children with brain injuries may appear as if there is nothing wrong with them, their internal brain injuries are very real and may or may not improve over time. Research indicates that the first year following the brain injury is the most important in terms of providing instructional services and therapies. It is in this period, that researchers believe the most important healing takes place, and it is critical to the student's future rehabilitation [5]. Two of the most common reasons for problems after a head injury are as follows:  Overestimating or underestimating the cognitive and behavioral abilities and limitations of the injured individual  Failure to understand the practical implications of deficits. Expecting too much from the injured individual frequently causes significant behavioral problems; expecting too little may also cause behavioral problems but, more importantly, limits recovery and the acquisition of new skills. At either extreme, the stress experienced by family members is exacerbated and increases over time [6]. Since brain injured children frequently have difficulty learning new information and generalizing new skills from one environment to another, the most effective learning programs occur in the home setting where old learning is maximized [7]. III. A SUPPORT FOR GESTURE BASED EDUCATIONAL SOFTWARE. The learning occurs at an extremely slow rate after brain injury: it may take thousands of trials to acquire new information and to be able to retrieve it reliably. Few family members have either the patience or the time to present the same material thousands of times. This is one of the reasons why educational software’s are increasingly used in cognitive retraining: they will present the same material in exactly the same way as many times as is necessary without becoming frustrated, angry or bored [8]. ISSN : 0975-3397 790 Er. Zainab Pirani et. al. / (IJCSE) International Journal on Computer Science and Engineering Vol. 02, No. 03, 2010, 790-794 Educational software can also be regarded as a remedial tool: in such cases, rehabilitation and not simply education is the main task to be performed and the choice of remedial tools (including software) should be founded on both a solid theoretical basis and on clinical findings [9]. This will affect not only the choice of products but also the evaluation of the results obtained. The intuitive gesture-based computing is gradually leading to new innovative kinds of teaching or training simulations that operate like their real-world counterparts [10]. Gestures can originate from any bodily motion or state but commonly form from the face or hand. Gesturebased computing allows humans to interface with a specific machine and interact naturally without any mechanical devices, thus children with brain injuries can overcome their disabilities [11]. A. Prototype Development An iterative design approach was used to develop the first prototype. Twelve children suffering from Acquired Brain Injuries were identified who helped us to design and evaluate our educational system. Thirty relatives and caregivers also agreed to take part as advisors and evaluators for the project. As a first step, we ask the group to comment on the content of the system. Ideas for themes the system includes are mathematical problems, puzzles, lexicon, and phonological awareness. The system was developed using Macromedia Director. Director was chosen because it is cross-platform development package that allows rapid application development of complex multimedia systems. It has its own programming language, which makes it interactive and allows us to connect to the database. The complete system was given the name GESBI standing for Gesture Based Educational Software for children with Brain Injuries. B. Description of interfaces Consultation with experts on Acquired Brain Injuries had indicated that the interface must be as simple as possible at the same time attractive and encourage interaction [12]. One problem with children with brain injury have is inability to cope up with too many items that compel attention. In this state, the child always focuses on one item and stay with it not being able to scan easily with other possibilities. To cope up with this the background has muted colors. And at a time only one item will be displayed. In order to keep interaction simple hand gestures were used. At any time, disabled child can exhibit his hand doing a specific gesture via fingers of his hand in front of a video camera linked to a computer. Figure 1. Screen Shot of GESBI C. Content of the system Based on the feedback of our advisors screen layout and content were designed as shown in Figure 1. It has four sections like maths, puzzle, lexicon and phono. The mathematical problems related only to subtraction, counting, finding quantity was added. In puzzles section – sudoku, crosswords were added. The lexicon section deals with day to day activities which a child performs like brushing of teeth which is broken down as picking of brush, applying paste, brushing teeth, putting brush back to its stand etc The Phonological awareness deals with the pronunciation of each and every alphabet and many words to increase the vocabulary plus also help child to pronounce it again and again as the brain injury suffers lot in pronunciation and grasping words. The system allows the child to select among the four sections via gesture of 1, 2, 3 or 4. Let us consider that child selects section one i.e. Math, so he or she can make use of gesture 1 of his finger. The gesture recognition algorithm‘KONCERN’ [1] is rotation, translation and scale invariant. So it is very helpful for a young child that he can select whichever finger he wants and whatever position of the finger may be as shown in Figure 2. Figure 2. Different forms of Gesture one Once the section of Maths is selected, GESBI enters second screen, which is as shown in figure 3. Now based on the desire of child he will go for sub sectionn of subtraction, counting of numbers or finding quantity via gesture of finger one, two or three. If gesture one is detected by GESBI then child can enjoy doing subtraction as shown in the figure 4. If gesture two or three is detected by GESBI then child can enjoy quantity and counting sub section as shown in the figure 5 and figure 6. ISSN : 0975-3397 791 Er. Zainab Pirani et. al. / (IJCSE) International Journal on Computer Science and Engineering Vol. 02, No. 03, 2010, 790-794 Figure 3. Math Section Figure 4. Subtraction sub section